EDA
We can first visualize the data after dimension reduction with zinbWave. We then use t-SNE and color the cells with the allen labels. Since we use various values for the k parameter, we plot two representations with the most extreme values of k.

Reduction in the number of clusters

Improvement in ARI


LS0tCmF1dGhvcjogIkhlY3RvciBSb3V4IGRlIELDqXppZXV4IgpkYXRlOiAnYHIgZm9ybWF0KFN5cy50aW1lKCksICIlZCAlQiAsICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2RlcHRoOiAyCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIGNvZGVfZG93bmxvYWQ6IFRSVUUKICAgIApwYXJhbXM6CiAgZGF0YXNldDogIjEweF92M19jZWxsc19NT3AiCiAgdGl0bGU6ICJBbmFseXNpcyBvZiB0aGUgU01BUlRlcl9jZWxsc19NT3AgZGF0YXNldCIKLS0tCi0tLQp0aXRsZTogYHIgcGFyYW1zJHRpdGxlYAotLS0KCmBgYHtyIGxvYWQgcGFja2FnZXMsIGluY2x1ZGU9Rn0KbGlicmFyeShrbml0cikKb3B0c19jaHVuayRzZXQoCiAgZmlnLnBvcyA9ICIhaCIsIG91dC5leHRyYSA9ICIiLCB3YXJuaW5nID0gRiwgbWVzc2FnZSA9IEYsCiAgZmlnLndpZHRoID0gNSwgZmlnLmFsaWduID0gImNlbnRlciIsIGVjaG8gPSBGCikKbGlicyA8LSBjKCJoZXJlIiwgImRwbHlyIiwgImdncGxvdDIiLCAidGlkeXIiLCAic3RyaW5nciIsICJyZWFkciIsICJjb3dwbG90IiwKICAgICAgICAgICJjbHVzdGVyRXhwZXJpbWVudCIsICJtY2x1c3QiLCAiUkNvbG9yQnJld2VyIiwgInByb2dyZXNzIiwgIkR1bmUiLAogICAgICAgICAgInBuZyIsICJwdXJyciIpCnN1cHByZXNzTWVzc2FnZXMoCiAgc3VwcHJlc3NXYXJuaW5ncyhzYXBwbHkobGlicywgcmVxdWlyZSwgY2hhcmFjdGVyLm9ubHkgPSBUUlVFKSkKKQpybShsaWJzKQp0eXBlIDwtIGZ1bmN0aW9uKGRhdGFzZXQpIHsKICBpZiAoc3RyX2RldGVjdChkYXRhc2V0LCAiU01BUlQiKSkgcmV0dXJuKCJTbWFydC1TZXEiKQogIGlmIChzdHJfZGV0ZWN0KGRhdGFzZXQsICIxMHhfdjMiKSkgcmV0dXJuKCIxMFhfVjMiKQogIGlmIChzdHJfZGV0ZWN0KGRhdGFzZXQsICIxMHgiKSkgcmV0dXJuKCIxMFgiKQogIGlmIChzdHJfZGV0ZWN0KGRhdGFzZXQsICJSZWdldiIpKSByZXR1cm4oIlJlZ2V2IikKfQp0eXBlIDwtIHR5cGUocGFyYW1zJGRhdGFzZXQpCm1lcmdlcnMgPC0gcmVhZFJEUyhoZXJlKCJkYXRhIiwgIkR1bmUiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYXN0ZTAocGFyYW1zJGRhdGFzZXQsICJfbWVyZ2Vycy5yZHMiKSkpCmBgYAoKIyBFREEKCldlIGNhbiBmaXJzdCB2aXN1YWxpemUgdGhlIGRhdGEgYWZ0ZXIgZGltZW5zaW9uIHJlZHVjdGlvbiB3aXRoIHppbmJXYXZlLiBXZSB0aGVuIHVzZSB0LVNORSBhbmQgY29sb3IgdGhlIGNlbGxzIHdpdGggdGhlIGFsbGVuIGxhYmVscy4gU2luY2Ugd2UgdXNlIHZhcmlvdXMgdmFsdWVzIGZvciB0aGUgayBwYXJhbWV0ZXIsIHdlIHBsb3QgdHdvIHJlcHJlc2VudGF0aW9ucyB3aXRoIHRoZSBtb3N0IGV4dHJlbWUgdmFsdWVzIG9mIGsuCgpgYGB7ciB0LVNORSBwbG90cywgZmlnLndpZHRoPTE4LCBmaWcuaGVpZ2h0PTl9CnBsb3RzIDwtIGxpc3QuZmlsZXMoaGVyZSgiRmlndXJlcyIsICJFREEiKSkKcGxvdHMgPC0gcGxvdHNbc3RyX2RldGVjdChwbG90cywgcGFyYW1zJGRhdGFzZXQpXQpwbG90cyA8LSBwbG90c1tzdHJfZGV0ZWN0KHBsb3RzLCAicG5nIildCnBsb3QubmV3KCkKcGxvdC53aW5kb3coMDoxLCAwOjEpCnJhc3RlckltYWdlKHJlYWRQTkcoaGVyZSgiRmlndXJlcyIsICJFREEiLCBwbG90c1sxXSkpLCAwLCAwLCAuNSwgMSkKcmFzdGVySW1hZ2UocmVhZFBORyhoZXJlKCJGaWd1cmVzIiwgIkVEQSIsIHBsb3RzW2xlbmd0aChwbG90cyldKSksIDAuNSwgMCwgMSwgMSkKYGBgCgoKIyBHZW5lcmFsIGNvbW1lbnRzCgpXZSBoYXZlIHRoZSBmb2xsb3dpbmcgd29ya2Zsb3cKCmBgYHtyIHdvcmtmbG93LCBmaWcuaGVpZ2h0PTYsIGZpZy53aWR0aD0xMH0KcGxvdC5uZXcoKQpwbG90LndpbmRvdygwOjEsIDA6MSkKcmFzdGVySW1hZ2UocmVhZFBORyhoZXJlKCJFeHBsYWluYXRpb25zIiwgIldvcmtmbG93LnBuZyIpKSwgMCwgMCwgMSwgMSkKYGBgCgpJbiB0aGUgU2V1cmF0IGNsdXN0ZXJpbmcsIHRoZXJlIGFyZSB0d28gcGFyYW1ldGVycyB0byBjaG9vc2UgZnJvbTogdGhlIHJlc29sdXRpb24gYW5kIHRoZSBrLnBhcmFtICh1c2UgaW4gYSBrbm4gc3RlcCkuIEZyb20gdGhlIHNldXJhdCBoZWxwIGZpbGUsIGZvciB0aGUgcmVzb2x1dGlvbiBwYXJhbWV0ZXI6IHVzZSBhIHZhbHVlIGFib3ZlIChiZWxvdykgMS4wIGlmIHlvdSB3YW50IHRvIG9idGFpbiBhIGxhcmdlciAoc21hbGxlcikgbnVtYmVyIG9mIGNvbW11bml0aWVzLgoKV2UgcGlja2VkIGEgdmFsdWUgb2YgMS42IGFzIHdlIHdhbnQgbWFueSBjbHVzdGVycyB0byBzdGFydCB3aXRoLiBXZSBhbHNvIHBpY2sgayA9IDUwLiBUaG9zZSB2YWx1ZXMgc2VlbSBpbnRlcm1lZGlhdGUgaW4gdGVybSBvZiBBUkkgd2l0aCBvdGhlciBwYWlycyBvZiBwYXJhbXRlcnMgKHNlZSBzZWN0aW9uIFNldXJhdCBBUkkgcGFyYW0pCgpXZSBhbHNvIGNvbnNpZGVyIHR3byB3YXlzIG9mIGNvbXB1dGluZyB0aGUgQVJJIGJldHdlZW4gUlNFQyBhbmQgb3RoZXIgbWV0aG9kcy4gRWl0aGVyLCB3ZSBmb3JjZSBSU0VDIHRvIGFzc2lnbiBhbGwgY2VsbHMgdG8gYSBnaXZlbiBjbHVzdGVyLCBvciB3ZSBvbmx5IGNvbXB1dGUgdGhlIEFSSSBiZXR3ZWVuIFJTRUMgYW5kIHRoZSBvdGhlciBtZXRob2RzIG9uIHRob3NlIGNlbGxzIHRoYXQgUlNFQyBkbyBjbHVzdGVyLiBOb3RlIHRoYXQgaW4gdGhlIHNlY29uZCBjYXNlLCBvdGhlciBwYWlycyBvZiBtZXRob2RzIGFyZSBjb21wYXJlZCB1c2luZyBhbGwgY2VsbHMuIFdlIGRlbm90ZSBhcyBSc2VjVCB0aGUgY2x1c3RlciBhc3NpZ25lbWVudCB3aGVyZSBhbGwgY2VsbHMgYXJlIGFzc2lnbmVkIChpLmUgUnNlYyBUb3RhbCkuIAoKVGhlIEFSSSBtZXJnaW5nIG1ldGhvZCB3b3JrcyBhcyBmb2xsb3cuIFdlIGl0ZXJhdGUgb3ZlciBhbGwgcGFpcnMgb2YgY2x1c3RlcnMgZm9yIGV2ZXJ5IGNsdXN0ZXJpbmcgbWV0aG9kLiBGb3IgZWFjaCBwYWlyLCB3ZSB0cnkgdG8gbWVyZ2UgdGhlIHR3byBjbHVzdGVycyBhbmQgc2VlIGhvdyBpdCBpbXByb3ZlcyB0aGUgQVJJIHdpdGggdGhlIG90aGVyIG1ldGhvZHMuIFdlIHRoZW4gbWVyZ2UgdGhlIHBhaXIgdGhhdCBpbXByb3ZlcyB0aGUgQVJJIHRoZSBtb3N0LiBXZSBzdG9wIHdoZW4gdGhlIEFSSSBjYW5ub3QgYmUgaW1wcm92ZWQgYW55bW9yZS4KCkluIHRoZSBnZW5lcmFsIGNhc2UsIHdlIHBlcmZvcm0gdGhlIEFSSSBtZXJnaW5nIHdpdGhvdXQgdGhlIGFsbGVuIGxhYmVscyBhbmQgd2UgdGhlbiB1c2UgaXQgYXMgYSBjb21wYXJpc29uLiBBIHF1aWNrIG92ZXJ2aWV3IG9mIGhvdyB0aGUgYWxnb3JpdGhtIHBlcmZvcm1zIHdpdGggdGhlIGFsbGVuIGxhYmVscyBpcyBzZWVuIGF0IHRoZSBlbmQuCgojIFJlZHVjdGlvbiBpbiB0aGUgbnVtYmVyIG9mIGNsdXN0ZXJzCgpgYGB7ciBJbXAgbm8gYWxsZW59CnBsb3RQcmVQb3N0KG1lcmdlcnMpCmBgYAoKIyBJbXByb3ZlbWVudCBpbiBBUkkKCmBgYHtyIEFSSSBpbXAgbm8gYWxsZW4gY2VsbH0KcGxvdF9ncmlkKAogIHBsb3RBUklzKG1lcmdlcnMkaW5pdGlhbE1hdCkgKyBnZ3RpdGxlKCJCZWZvcmUgbWVyZ2luZyIpLAogIHBsb3RBUklzKG1lcmdlcnMkY3VycmVudE1hdCkgKyBnZ3RpdGxlKCJBZnRlciBtZXJnaW5nIikKKQpgYGAKCgpgYGB7ciBBUkkgdHJlbmQsIGZpZy53aWR0aD05fQpBUkl0cmVuZChtZXJnZXIgPSBtZXJnZXJzKQpgYGAK